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Over the last 20 years, a small group of investigators have attempted 
to identify those teaching behaviors which were related to student achieve- 
ment. Their purpose is to learn how to improve the quality of the learning 
that takes place in school. Although the major focus has been the improve- 
ment of cognitive gains — usually in reading and mathematics — there have also 
been concern with affective outcomes, for example, liking for school and 
self-concept. Given the current public concern with the mastery of basic 
skills and the decline in test scores, this continued study of teaching 
behavior remains important. 

This research has traditionally focused on identifying which teacher 
behaviors (e.g., teacher talk, number of higher order questions, clarity 
of presentation) were correlated with student achievement gain. This line 
of research has not been particularly productive, and teacher behaviors 
which were offered with confidence a few years ago are no longer seen as 
clearly related co student achievement gains. 

The studies which have been published since 1973 have been more 
promising in terms of suggesting productive instructional strategies; it 
is these studies which are reviewed here. The major changes in thinking 
may be summarized as follows : 

1) An increased focus on student variables such as content covered 
(or opportunity to learn) and student attention to relevant academic 
activities. 



th:S oocument has been «ep«o- 
ouceo exactly as received prom 
the person or organization origin- 
ATING IT POINTS OP VIE W OR OPINIONS 
STATEO 00 NOT NECESSARILY REPRE- 
SENT OPPlClAL NATIONAL INSTITUTE OP 
EOUCATION POSITION OR PCLlCY 



2 



4 



2) A convergence of results supporting several components of a model 
of "direct instruction," 

3) New information on the proportion of student time spent in seat- 
work and in discussion, with implications for the role of the teacher. 

Cautions and Considerations 

First, the reader should note that the research to be reported is 
limited to instruction in basic skills — reading and mathematics achievement 
gain — in U.S. grades 1 through 5 (ages 6 through 10). The learning of 
these basic skills is termed "didactics" (Broudy, 1972) in that learning 
takes place in a linear, logical, sequential manner. 

Broudy believes that most of the standard school subjects fall into 
the domain of didactics, and that standard achievement tests are the obvious 
measures of didactics. Although one can make logical separations of 
elementary skills such as decoding and arithmetic computation and more 
advanced skills such as arithmetic problem solving and reading comprehen- 
sion, both sets of skills would be termed didactics by Broudy — and, indeed, 
research to dale with elementary school cliildren has shown these two sets 
of skills to be highly correlated (.8 to .9). 

This focus on didactics does not reflect a lessoned concern with 
students attitudes and feelings, hunione interact ioy^is In the classroom, or 
the teaching of heuristics. These issues, and the distinctions between 
heuristics and didactics will be discus ad later in this paper. 

A second caution concerns the overall quality of these studies, many 
of which contain methodological problems or flaws. For example, the large 
number of variables studied — in some cases over 500 in a single study — 
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means that many significant correlations were obtained by chance. Despite 
these problems, a consistency in the conclusions across studies lends 
credence to their validity; it is this consistency which is emphasized in 
this paper. Furthermore, most of these investigators are currently engaged 
in experimental studies to validate and probe their correlational results. 

The third and major caution is the need to proceed with caution in 
inplementing these findings into teacher training programs or into evaluative 
checklists for teachers. Rather than quickly adopting another round of 
dictuES on "teachers should" as we did in the late 60 's and early 70' s, 
it would be preferable to spend time thinking about and discussing the 
results, their implications, and possible alternatives. For example, 
although the present findings can permit a wide diversity of teaching 
styles, soma results may seem grim and overbearing. It may be possible, 
with due reflection, to devise instructional alternatives which are more 
appealing but equally effective. 

Academic Engaged Time 
Recently, researchers have moved from a primary concern with teacher 
behaviors to a number of other instructional variables. Although teacher 
behavior is still clearly important, attention has recently been focused on 
student variables. This emphasis on the student has been influenced by the 
ideas of Carroll (1963) and Bloom (1976). Two particularly useful student 
variables arc 1) content covered' or "opportunity to learn" and 2) student, 
attention or engagement. These two variables have been combined here into 
the concept of academic engaged ti me. This concept, as originally developed 
by the staff at the Far West Laboratory (Berliner et al. , 1976) refers to 



the time which a student spends engaged in academically relevant material 
which is of a moderate level of difficulty. Although content covered, 
student attention, and academic engaged time are cloj;ely related, content 
covered is the more critical variable, since it is closer to the outcome 
measures. However, because it is easier to obtain data on academic engaged 
time, most of the results reported below concern academic engaged time. In 
this paper, academic engaged time will be used as a tentative proxy for 
content covered, but the prime importance of content covered, particularly 
for didactics, should be emphasized. 

Content covered . Content covered or "opportunity to learn** (Carroll, 
196?) has been studied in many ways, including inspecting the content of 
textbooks ussd IPidgeon, 1970), asking teachers to indicate the percent of 
students who have had an opportunity to learn each item on the test (Husen, 
1967; Con.ber and Keeves, 1973; Chang and Rathr,, 1971), counting the number 
of pagis of the c':.amon textbook covered during the semester (Good, Grouws, 
and Beckerman, in press), coding the content in a short presentation which 
was relevart to the exam questions (Armento, 1977; Rosenshine, 1971; Shutes 
1969), counting the number of words which the teacher attempted to teach 
(Beez, 1968; Carter, 1969; Bro^^m, 1969; Barr, 1973) counting the amount of 
mathematics problems covered (McDonald, 1975), comparing the results of 
different curriculum programs on general and on curriculum-related post- 
tests (Walker and Schaf farzick, 1974) and coding the level of the workbook 
... the students completed just before they took a post-test (Rosenshine, 1976) 
In all studies but one (Brown, 1969^ significant relationships were 
found between content covered and student achievement gain. Moreover, 



these correlations were asuplly larger than those obtained for teacher 
behavior variabLjr \s M :Donald (1975, p. 27) noted, "If students have 
not been taught, . .some. • .content or procedure, they simply do not do well 
on those poritions of the test relevant to that topic," 

Although this conclusion sounds like common sense, investigation of 
content covered has not been common practice in educational research: 
measures of content covered have seldom been used in the major studies 
of classroom activities and student progress, (e»g,, those by Flanders, 
by Brophy and Evertson, Stallings and Kaskowitz; by McDonald, by Soar, 
by Solomon and Keadall, or by Bennett, 

Student attention or engagement . Student engagement with material 
or attention is obviously necessary for student learning. Despite the 
difficulties of measuring attention (e.g., do reading, writing, listening 
to the teacher, or following another reader count equally as "attention"?) 
and the crude and varied procedures used to gather this data. Bloom (1976) 
found clear and consistent results in the research he reviewed on student 
attention. In the fifteen studies he reviewed. Bloom found that the cor- 
relations of student attention with student gain v/ere about .40 when the 
student was the unit of analysis, and about ,52 when the class was the unit. 
Stallings and Kaskowitz coded students as attending only when they were 
obviously working on reading or mathematics activities and not when they were 
engaged in construction or dramatic play. In that study, time spent involved 
in reading or mathematics yielded higher correlations (.3 to .6) with 
achievement than any oJ the other coded teacher or student behaviors. 
Similarly, the use of texts, workbooks, or instructional materials yielded 



positive results, but no nonacadeciic activity yielded positive correlations 
with reading and ir.nthenatics achievement* 

These two variables of content covered and student attention have 
yielded the highest and most consistent correlations with achievement gain 
of any of the classroom variables studied to date. The messap;^ is: what 
is not taught and attended to in a^cademic areas is not learned. 

A fairly dramatic example of the advantage of studying academic 
engaged time can be obtained from the work of Stallings and Kaskowitz (1975). 
Every fifteen minutes, as part of their observational study, they took a 
"snapshot** of what every child in the class was doing and the group he 
was in. Their results on the percentage of time that the average class was 
productively engaged in reading or math are presented in Table 1 with results 
given for each Follow Through Model. The rank order for achievement gain 
in each model is also given. One notes, particularly in reading, a strong 
correspondence between the relative time spent engaged in reading or math 
activities and achievement gain. (The results on the Non-Follow Through 
or control classrooms are also revealing, showing that children were engaged 
in reading or math relatively frequently in those ^'traditional" settings). 



One notes from Table 1 that in reading, children in the top three models 
in achievement gain were spending about fifty percent more time in academic 
activities than were children in the lox^zer three models. 

It should be emphasized that the amount of time allocated for content 
coverage may be quite different: i'rorn academic engaged time; allocated time 
may include activities that do not rr-cuire actual involvement in academical^ 
relevant material. Thus, we find ch^t in studies which consider only 
allocated time, most of the results tend to be non-significant (Smith, 1976; 



Table 1 

Percent engaged time and achievement gain by Follow Through Program 







1st 


grade 






3rd 


grade 


Program 


Reading 


Math 


Reading 


Math 


Oregon 


54* 


1* 


27 


5 


58 


2 


V 1 


Kansas 


59 


2 


22 


1 


59 


3 


22 3 


Non-Follow 
















Through 


50 


3 


18 


3 


48 


1 


22 2 


Bank Street 


40 


6 


16 


4 


38 


7 


20 4 


High/Scope 


40 


5 


20 


8 


46 




16 


Arizona 


40 


4 


14 


6 


42 


6 


16 7 


Far West 


39 


8 


14 


7 


41 


4 


18 6 


EDC 


29 


7 


14 


2 


37 


5 


24 5 



* In each, the first column ia average class percent engaged time 
and the second column is achievement gain ranV 

From Stallings and Kas'cowitz, Appendix pages R-3 and and from Appendix 

0» Appendix 0 is a separate document which contained the data on Non-Follow 
Through classrooms . 

Rank Order Correlations 



1st grade reading: ^99 
math: .17 



3rd grade ^reading: ♦80 
math: .78 
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Welch and Bridghma, 1968; Guthrie et al. , 1976). 

Despite the importance of academic engaged time, it should be 
emphasized that a teacher is not obligaged to have all of her students 
100 percent engaged all of the time. The critical variable is the total 
academic engaged minutes which occur during the day. 

Consider, for example, tx^o teachers whom we studied in Urbana. The 
teachers were both personable, warm, competent women in their early 30' s 
who were using the same materials—the McGraw Hill Prop.rammed Readers— 
with similar children~;:he sons and daughters of college professors and 
graduate students. The first teacher allotted 30 minutes per day for reading, 
and ran a tightly structured classroom in which students were recorded as 
engaged 80 percent of the time. She maintained this high percentage by 
being very alert to student engagement and using frequent social control — 
such as saying "Johnny, can I help you?''"When a student wasn't working. 
The second teacher allotted 60 minutes to reading, but ran a looser clat^s- 
room and was less insistent on high engagement. The students in the second 
teacher's class averaged acadfimi ; engagement only '65 percent of the time. 

If one computes the number of engaged minutes, then the first teacher 
averaged 24 engaged minutes per day (80 percent of 30 minutes) whereas the 
second teacher averaged 39 engaged minutes because she allotted more time. 
These differences were reflected in our results: during the ten weeks of 
our study, the students in the second classrootr. covered proportionately 
more books in the McGraw-Hill readers than did the equally bright children 
in tha first classroom. Thus, a teacher is not obligated to maintain high 
student engagement at all times: what is more critical is the total number 
of academically engaged minutes and the amount of content covered. 
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Unfortunately, it is not yet known just hov/ many acaclenic engaged 
minutes or how much content coverage is sufficient or necessary. Indeed, 
we have little data on the academic engaged time actuidly occurring in 
schools. One source of such information is the 'iKxiiy by McDonald (1976). 
In this study, observers coded, among other things, the "productive minutes" 
or number of miautes per day of academic enjjagcd time for target students. 
The results show that students were enf;aged from about 70 percent of the 
allotted time — a figure which appears consistent with other studies. But 
if one looks at the median productive minutes per day , one finds less than 
30 productive minutes a day for reading in either grade 2 or grade 5, and 
less than 25 minutes per day in math. Such figures suggest that students arc 
currently not being overloaded with practice in basic skills. 

Direct Instruction 
The s^^icond concept, direct instruction, is related to the concept of 
academic engaged time. The term is relatively new, and apparently was 
developed independently by a number of researchers in the last three years. 
The meaning of the term is still >^eing developed, and the definition is 
still loose. I use it to refer to those activities which are directly 
related to making progress in reading and mathematics and to those settings 
\;hich promote those activities. Essentially, it refers to those variables 
which promote content covered and academic angaged time. A number of 
instructional variables fit the concept of direct instruction. 



10 



9a 

To give an overview of the results, direct instruction refers to high 
levels of student engagement witliin teacher-directed classrooms using 
sequenced, structured mater.^, ais. As developed belov/, direct instruction 
refers to teaching actl v t ^.^s focuaed on academic matters where goals are 
clear to students; tii .ulc;atea for instruction is sufficient and 
continuous; content coverage Its extensive; student performance is monitored; 
questions are at a low cognitive level and produce many correct responses; 
and feedback to students is immediate and academically oriented. In direct 
instruction, the teacher controls instructional goals, chooses material 
appropriate for the student's ability level, and paces the instructional 
episode. Interaction is characterized as structured, but not authoritarian; 
rather, learning takes place in a convivial academic atmosphere. 

Let us understand the task. The goal is to move the students through 
a sequenced set of material or tasks. Such materials are common across - — 
classrooms, and have a relatively strong congruence with the tasks on the 
achievement tes t s . 

i^ote that v/e are limiting the term direct instruction to didactic ends, 
that is, for instruction tov;ard rational, specific, analytic goals which 
appear on the reading tests we* are currently using. These didactic exer- 
cises predominate both the reading and math materials currently in use as 
well as the reading and math tests. 

How, let us turn to tiie results ^ The choice of instructional variables 

selected for this review is illuminating and shows how we hcve progressed 

in the last few years. V/e no longer iiave lists of promising variables such 

as clarity or criticism. Instead, the results are organized into seven topic; 

which appear to reflect the organization and management of classrooms more 

appropriately. These topics are: 

the role of the teacher seatwork 
student choice discussion 

gro up ings ^ a tmo sp 1 le re 
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Table 2 



Allocated and productive minutes in reading and math (Afcer McDonald, 1S76) , 



Reading 

Median allocated minutes 
Median productive minutes 
Percent productive time 
Mean allocated -ninutes 
Mean productive minutes 
Percent productive time 



Grade 2 
35.64 
28.00 
78.00 
63.30 
47.72 
75.00 



Grade 5 
33.75 
22.21 
66.00 
55.00 
42.04 
76.00 



Mathematics 

Median allocated minutes 
Median productive minutes 
Percent productive time 
Mean allocated minutes 
Median productive minutes 
Percent productive time 



18.27 
15.15 
83.00 
37.54 
28.82 
76.00 



30.92 
23.95 
77.00 
48.00 
35.44 
74.00 
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Tea Cher- centered. The studies by Soar, Stallings and Kaskowitz, 
and Solomon snd Kendall showed that the teachers wh o most succ essfully 
promoted achievement gain played the role of a strong leader; that is, they 
directed activities without giving their students choices, approached the 
subject matter in a direct, business-like way, organized learning around 
questions they posed, and occupied uhe center of attention. In contrast, 
the less successful teachers made the students the center of attention, 
organized learning around the studenur,' owu problems, and joined or par- 
ticipated in student activities. 

Student choice of activity . The converse oF teacher-centered, student 
choice of activities—yielded negative results. Classrooms which were 
organized so that students have a great deal of choice about the activities 
they will pursue are usu^illy ones with lower academic engaged time and lower 
achievement- For example, in the study by Soar (1973), student free choice, 
student limited choice and free v;ork groups were associated wi\:h lessened 
achievement gain. For Stalliugs and Kaskowitz (1974), child selection of 
seating and work groups and a v:ide variety of concurrent activities yielded 
similar consistent negative correlations with achievement gain. These 
two studies were lir.:ited to children of low SES backgrounds. However, 
Solomon and Kendall (1976) obtained the jsa^ results in their study of 
children in thirty foUiTth prada suburban classrooms. In this study, 
classrooms in wiiich scudr>ntri chose their activities and followed theit 
own interests, were runponsiMe for class planning, and were not teacher 
dependent were also tho classrooms m\±c\\ were characterized by rowdiness, 
shouting, noise, and disorderliness . In Solomon and Kendall's study this 
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permissiveness, spontaneity, and lack of control classroom characterization 
were negatively related, not only to achievement gain, but also to positive 
growth in creativit", inquiry, writing ability, and self-esteem for the 
students in those classrooms. Similarly, in an. unpublished study by Good 
and Beckerman on student engageir.euc ia six 6th grade middle class and lower 
classrooms, students were more engaged when the teacher assiwed the work 
than when the students chose the work. 

These results do not mean that all attempts at infortnality were dis- 
astrous, but rather that extremes of student autonomy and self-direction 
were usually associated with less task orientation and student engagement 
and consequently, less student gain on all measures. A possible explana- 
tion of these results is that when students have many choices, among a 
variety of activities and are not in groups being supervised by a teacher, 
they are particularly susceptible to distraction and_find it difficult to 
stay on task for a productive period of time. Evidence for this explanation 
of distractability was obtained in a thoughtful ecological study by Kounin 
and Doyle (1975). They found that preschool children were quite attentive 
at construction tasks at their desks when the materials were ari-ang2d 
beforehand. But when students had to frequently Save their seat to 
obtain materials, they were highly susceptible to distraction. 

Grouping stu d ents for learning . The studies of primary grade class- 
rooms point to the need for adult monitoring and supervising of student 
activities. Stallings and Kaskowitz found that time snent working v/ith 
only one or two students, was negatively related to class achievement gain, 
whereas time spent working with small groups (three to seven students) or 
with large groups was consistently positively related to achievement. 
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Similarly, classes which had a wide range of concurrent activities also had 
lower achievement scores, suggesting that teachers were unable to supervise 
such a variety of activities. Likewise, Soar (1973) discovered that when 
studenva worked in groups under adult supervision, correlations with 
achievement were positive and often significant. On the other hand, when 
small groups met without an adult, correlations between this grouping 
pattern and achievement were negative and often significant. A simple fact 
may be inferred from the studies cited: given that many students do not 
engage in on-task behavior unless a teacher or another adult is monitoring 
their academic activities, the use of large group settings allows for mora 
adult supervision. Although many educators advocate working with one or two 
such an arrangement precludes adequate supervision for the remaining children. 
As a result, most of the children have less academic engaged time. 



Management . Management is a critical problem which has not received 
enough attention. Classes with poor management usually have low academic 
engaged time. For example, teacher procedur?il. contacts with students were 
negatively related to achievement gain in studies by Brophy and Evertson 
and by Good and Grouws, suggesting that procedural activities do not repre- 
sent academic engaged time. 

A major objective for future research is learning how to manage 
students when they are working alone. Although informal classrooms typically 
have a large number of students working alone, students in all classrooms 
spend from 50-70 percent of their time vor^:ing alone. For example, in 
formal classrooms where a single teacher is responsible for three reading 
groups, two-thirds of the students will be expected to work alone at all 
times. We need to know how to increase academic engaged time for students 
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left on their own in such settings. There has not been sufficient explora- 
tion of the teacher behaviors that help keep students occupied. A few 
tentative but sensible suggestions emerged from a study of more-effective 
and less-effective classrooms in second and fifth grade reading or mathematics 
(Tikunoff, Berliner and Rist, 1975). In that study, effective teachers 
allocated sufficient time to a. . -'ivity aiv! were not bound to the clock 
as a guide for starting or stopping activities. Similarly, time allocated 
to a subject was continuous, without abrupt shifts from instruction to 
•behavior management to announcements to large group instruction. 

Students work i ng privately . Questions and answers, discussion, and 
other types of oral classroom activities may not be so prevalent as once 
thought. In a recent study (McDonald, 1975) of 40 second grade and 40 
fifth grade classrooms in California, the dominant activity was seatwork 
which occurred from 42 to 62 percent of the time. (See Table 3). Good 
and Beckerman obtained similar results in six 6th grade classrooms, with 
seatwork also occurring 51 percent of the time. 

Note that "seatwork" as used by McDonald, by Good, or by me does 
not refer lo the limited experience of students copyi-ig on a ditto sheet. 
Rather, seatwork refers to those activities students are doing privately 
such as reading a book, completing a workbook, writing a report, or using 
special cassettes machines, or books. In reading and mathematics instruc- 
tion, students are proceeding through a fairly common series of linear 
tasks, and in dolnR these ta sks thev a ^ e interacting with materials much 
more than thev are interacting with a teacher . The current research shows 
(e.g., McDonald, 1976, Stallings and Kaskowitz, 1975) that unsupervised 
students are less academically engaged tlian supervised students. 



Table 3 



Summary of Time Spent (After ETS report to California Commission) 



Activity 



2nd grade 
Reading 



2nd grade 
Math 



5th grade 
Reading 



5th grade 
Math 



Seatwork 42.4 

Question and 

answer 14.8 

Discussion 6.2 

Lecture 1 • 3 

Recitation 9.1 

AV, game, 

demonstration . 10.0 

Unknown 16.2 



49.8 

14.7 
11.0 
0.0 
0.4 

12.8 
11.3 



56.6 

9.1 
10.2 
4.0 
6.1 

2.4 
8.8 



62.5 

11.4 
6.2 
3.3 
0.0 

6.5 
10.3 
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Therefore > a major problem is maintaining student engagement in seatwork. 
The solution which successful teachers appear to have worked out is to 
supervise seatwork in a large group setting, and to thus avoid working 
with smaller groups and individual students. Hopefully, other reasonable 
alternatives are available. 

In the organization and supervision of seatwork, the only question 
which has been studied is the advantage of teacher initiated versus student 
iikttiated contacts during seattjork. Studying middle class students, both 
Good and Grouws and Brophy and Evertson found that teacher initiated contacts 
during seatwork were negatively related to achievement gain, whereas student 
initiated private contacts were positively correlated with achievement gain, 
particularly when the teacher responded with process feedback. In these 
studies, there appears to be an advantage in the teacher being available to 
provide help when needed, i^o other studies were found which directly coded 
seatwork interactions. 

Verbal interaction . New issues and problems are emerging in the area 
of questions and discussion. 

1« Prevalance of discussion . Discussion may not be as prevalent 
as once thought. As noted from McDonald's study, all verbal behavior — 
discussion, drill, recitation, and lecturing may occur only about 20 
to 30 percent of the time. In most of these studies, the elementary 
teacher today is much more a manager of students on materials and tasks 
than she is a leader of discussion. 

2, Factual questions and controlled practi ce. The frequency of factual, 
single-answer questions is positively related to achievement gain in most 
of these studies. Stallings and Kaskwitz found that the frequency of 
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academically focused direct questions at the two lower levels of the Bloom 
Taxonomy resulted in increased acquisition of basic arithmetic and reading 
skills. They identified a factual question — student response — teacher 
feedback pattern as most functional. Using a sample of low-income students, 
similar to those in the Stallings and Kaskowitz's study, Soar (1973) also 
found that factors with high loadings from variables such as convergent 
questions, drill, or questions that have single answers usually correlated 
positively with achievement. The results suggest that a pattern of 
"controlled practice" consisting of factual questions, st:udeut academic 
responses, and adult academic feedback appears to be functional. 

itedley, in hit3 rcviai; (lv77) extrapolated tv;o typc3 of controlled practice 
pattcm.'J^ The uucceaaful pattern for low'SES children iu one x^ith simpla que^tioi 
a high percentage of correct answers, help to the student when he doesn't 
know the answer, and infrequent criticism. For high SES students, the 
successful pattern included harder questions and fewer correct answers 
(about 70 percent correct answers), little discussion and amplification of 
correct answers, criticism for Incorrect answers, and calling on another 
student when a student doesn't know the answer. A siiailar type of pattern 
had a positive but non-significant trend in the study by Solomon. A 
similar pattern, labeled "recitation" was most successful in moderate amounts 
of Soar's study of 5th grade middle classrooms. Thus, controlled practice 
appears to be generally functional. 

3. Higher order questions . Although teachers have been urged by 
teacher educators to ask higher cognitive level questions, recent research 
does not support this emphasis. In Stallings and Kaskox^^itz's study, the 
frequency of open-ended questions (i.e., questions high in the Bloom Taxonomy) 
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was negatively related to student achievement gain. In Soar's study, factors 
with loadings from variables like divergent questions and open-ended questions 
usually correlated negatively with achievement, 

_ In^two extremely well-designed experimental studies (Gall et al. , 1975; 
Stanford Program on Teacher Effectiveness, 1976). Teachers were semiscripted 
in that they were provided with scripts containing the questions to be asked 
and responses to be given. Overall, both studies found that classes where 
students were asked more recall questions did slightly better on the recall 
tests, whereas all classes did equally well on tests which included higher 
level questions no tnatter how many higher-level questions had been asked 
in class. In other words?, asking different numbers of higher-level questions 
had no measurable effect on essay performance or on tsats containing higher 
level questions. 

Perhaps there is confusion over what is meant by higher cognitive level 
questions. Questions that require a student to search the text and infer 
(e.g., ";Jhat words tell you that Mary felt sad?") are frequently coded as 
lower order because they have a single answer to be taken from the text, 
whereas questions of opinion (e.g., "How do you think Mary felt?") are 
frequently coded as higher level because they have move than one possible 
answer, although none of those answers are found In the texts. Therefore, 
the inferential "lower" questions, which require searching the texts may 
be more representative of academic engaged time. 

Many questions which are coded as "higher level" may really be personal 
and opinion questions. In the studies by Stallings and Kaskowitz, by Soar, 
and by Brophy and Evertson, open-endea questions, personal experience 
questions, and opinion questions were negatively correlated with achievement. 
Such results suggest that these questions are best categorized as outside 
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the content to be covered and do not represent academically engaged time. 
Similarly, questions of opinion about a text (e.g.,**How do you think Mary 
felt?") may represent non-academic questions. 

If one inspects the comprehension questions which appear In the basal 
series workbooks (see Armbruster, Stevens, and Rosenshine, 1977) one notes 
a general similarity between the types of questions which appear in those 
workbooks and the questions which appear on stai:dardized reading tests. In 
this sense, the workbooks are "practicing the test." From this point of 
view, it makes sense to expect that verbal activities which represent con- 
trolled practice would be positively related to achievement gain, whereas 
the other types of verbal activities would be unrelated to achievement gain 
because they are too far from the types of skills required for the test. 
Although verbal questions such as "what is a good title for this story" or 
"how would you summarize this paragraph" may be considered as lower level 
by many educators. It is these skills which appear on the tests; questions 
which are coded as interpretive or critical thinking simply do not appear 
on achievement tests. 

4. Other types of discussio n. Beyond controlled practice, or recita- 
tion, the advantages of other types of discussion is not clear. Two other 
types of questijning patterns appear across these studies: 

conv€:rcent questioning with amplification and elaboration 
open questions with encouragement of expression 
Both types have a slight negative trend with achievement gain when they 
appear. However, it is difficult to interpret even this finding because 
in the studies by Soar and Solomon, andpossibly by McDonald, we don't know 
whether such discussion even took i^lace during reading and math because the 
subject area was not identified in coding or was not used in forming variables 
for analysis. 
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5, Student Initiated talk . With the exception of the study by Good 
and Grouws., student initiated talk in discussion is not emerging as a 
significant correlate, student initiated contacts during seatwork, however, 
remain positively correlated with achievement gain. 

Atmosphere . An environment that stresses academic achievement, making 
use of many of the components of direct instruction discussed earlier, need 
not be coercive or aversive. In fact, classrooms which have high achievement 
gain frequently have a warm atmosphere. For example, in the ethnographic 
study by Tikunoff, Berliner, and Rist (1975) the hi.f^her achieving classrooms 
were observed as being convivial, cooperative, democratic, and warm whereas 
in the lower achieving classrooms there was more belittling, shaming of 
students, and use of sarcasm. Furthermore, competitiveness did not dis- 
tinguish between higher achievement and lower achievement. Other studies, 
such as that of Solomon and Kendall, also found that teacher criticism of 
student behavior, shouting, scolding, ridicule, and sarcasm were consistently 
negatively related to achievement gain. 

Tliese studies indicaLc that there is no need for teachers to be harsh 
and demeaning in order to obtain academic enga^^ed time, and that decent, 
humane, genuine interactions occur in many classrooms which are highly 
structured and teacher directed. 

Discussion 

A fairly clear and consistent pattern emerges from the studies cited. 
Student time spent engaged in relevant content apoears to be essential for 
achievement. Therefore, effective classroom teacbinp, of basic skills takes 
place in an environment characterized by an emphasis on academic achievement. 



Teaching behavior which is not diiectly aimed at furthering academic achieve- 
ment of the kind measured by standardized achievement tests will not result 
in much academic gain in these areas- Teachers who make a difference in 
students' achievement are those who put students into contact with cur- 
riculum materials and find ways to ke^r them in contact. Aithour^h non- 
academic activities, nonacademic questions, and a concern with a student's 
opinions and personal life may have motivational value, such activities do 
not in themselves represent academic engaged time. In addition, as 
laudable as it may seem to offer choices to students and to have them work 
on their unique, individualized assignments, such activities are usually low 
in academic engaged time. 

The more successful teacher is one who structures and selects the 
activities, who obtains a lar^e number of academic engaged minutes each 
day, who tends to a«k questions which have specific answers in a controlled 
practice format, who places students in groups where they are supervised 
by the teacher, c*nd who does this in a controlled but convivial classroom. 

Value questions . There is something grim about this picture of direct 
instruction: large j^ro?jps, teacher decision making, limited student choice 
of materials and activities , orderliness , factual questions , limited 
exploration of ideas, drill, and high percentages of correct answers. We 
immediately inquire about possible negative side effects. 

Although there are a few exceptions, such as some early Follow Through 
data from Stallings, and balance of data to this date does not show negative 
side effects. In tho two most recent Follc5w Through data, for Cohort III and 
IV, which entered school in the Fall of 1971, the t^^70 models which had the 
highest proportion of significant positive results on the Coopersmith 
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Self -Esteem Inventory were: Oregon's Direct Instruction and Kansas' 

Behavior Analysis — the two most structured, direct instruction programs. 

Similarly, in the study of middle class children by Solomon, the factor 

> 

labeled "control and orderliness'' was not only significantly related to 
achievement gain, but was significantly related to total gain in creativity, 
inquiry skill, writing quality, and self-esteem. In Bennett's study, the 
students in the informal classrooms do report better attitude toward school, 
but they also report more anxiety than students in other classrooms, a 
finding which matches Wright's report of lower achievement and higher test 
anxiety in informal classrooms. 

The image of the formal classroom as humorless, cold, regimented, 
military workplaces simply may not be true. The effective formal classrooms 
are "not^'coia'a^^ critical—McDonald found criticism was relatively rare in 
California classrooms, being concentrated mainly in tne less effective 
classrooms. Teachers in formal classrooms today are warm, concerned, 
flexible, a.id allow much more freedom of movement. But they are also task 
oriented and determined that children shall learn. 

One recalls some European classrooms in which children were sent at 
age four to learn to read from teachers who had failed at all other pro- 
fessions. They were taught long hours, in dingy rooms, and their first 
reading was to learn prayers~in a lanp;uage that v:as never spoken in the 
home. Out of this came a group that is particularly noted for thexr sense 
of humor and love of learning— I'm describing the schtel schools of the 
Jews of eastern Europe. 

But let us discuss the issue further by looking at didactic and heuristic 

instruction. 
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Didactic and heuristic Instruction , The above discussion, particularly 
the points on individualizing and student choice, remind us of the age old 
controversy in education between a model of didactic, formal, controlled 
instruction on one hand, and heuristic, informal, Inquiry, discovery instruc- 
tion on the other. The evidence here suggests that it is the formal model 
with its behavior-anelytic, detail-specific, teacher-directed, large group, 
and narrow questioning approach which is most effective for obtaining gains 
in reading and mathematics. The goal in such instruction is developing left- 
brained activities — the rational, analytic processes, and the approach mirrors 
the rational analytic goals. 

At the other end we have the more open model of instruction which 
focuses on inquiry, learner-choice, individualized work, learner Iriterests, 
and time for exploration and discovery. Such approaches suggest right- 
brained processes, and seem appropriate for developing intuitive and 
creative skills. It would seem that each approach may be most effective 
for its own end, and mixing approaches — such as usinf^ analytic, teacher- 
directed procedures to teach creative skills, or vice-versa — is not particu- 
larly effective. 

I have visited a small "free school" which use both approaches, but 
separately in their instruction. Mornings in these schools are spent in 
structured programs in reading, writing, and mathematics. The teachers 
make the assignments but the children complete them anjrwhere in the room 
in a relaxed and Informal manner. These assignments are from the same 
sequential, structured workbooks and readers that are used in traditional 
schools. Although each child works at his ovm task and at his own pace, 
no iDore than two activities, such as reading and writing, occur at the 
same time. Although the atmosphere is relaxed. Informal, and respectful. 
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the setting is large group, teacher centered, and structured. Alternoons 
are given to projects, exploracion, messing around, trips, and discussion. 
Fridays are for hobbies and crafts. Thus, the school teaches didactic 
goals, such as reading and math, in a didactic way; and spends the remaining 
time on more open activities. This approach seems intuitively appealing 
and sensible. I am not sure whether such an approach is possible on a large 
scale, or with children who come from homes with less verbal > activist 
parents. 

Suciraary. This paper has focused on content covered and student attention 
and engaged time. This represents a major shift from the past practice of 
looking at teacher variables such as affect, warmth, criticism, clarity, 
amount of teacher talk, enthusiasm or . probing. The focus is away from lists 

jf specific classrooms teacher behaviors to an emphasis on academic engaged 
minutes and the variety of procedures which might be used to obtain academic 
engaged minutes. There are no lists of essential teacher behaviors (although 

there are suggestions), nor is it claimed that any one type of teaching 
method or style is inherently superior. Thus, we can recognize the advantage 

of informal classrooms which are high in academic engaged time as well as 

the disadvantage of disorderly, formal classrooms (e.g., Marshall, 1977). 
The prlma i y goal of a teacher is obtaining "sufficient'' student content 



covered and academic engaged minutes. (Needed research on defining 
"sufficient" or "reasonable" will be discussed in the next section). But a 
teacher has a number of options. One would be to run a structured, ord^-rly, 
teacher-directed classroom, with an academic focus, and frequent monitoring 
and supervision of the students. The research suggests that such classrooms 
are high in percentage of engaged time per minute and can also be warm and 
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encouraging. However, a teacher could also choose a more flexible setting 
in which there is more individual work and/or more student choici. We 
know, that students in such settings tend to expend less engaged time per 
minute. Therefore, a teacher could use such settings only if. he or she 
increased the allotted time . This extra time could be obtained by decreasing 
some of the support and transitional activities. A third option would be 
for a teacher to critically examine his or her own teaching to determine 
how successful the current procedures are for obtaining academic engaged 
time. A teacher whose unique blend of procedures are obtaining sufficient 
academic engaged time need not change those procedures. 

Research Questions 
The m-^jor research question is that of discussing just how much more 
knowledge is needed in the area of instructional procedures for obtaining 
achievement gain in didactic areas such as redding and mathematics. I would 
hope that at some time we could consider this research area generally 
finished, and could turn more time and energy to instruction for heuristic 
or philetic ends. Yet, some major research issues remain and these are 
discussed in the hope that others ^-ill see theae aresj as worthy of study. 

Academic engaged ninutes and c ontent covered . The major research 
question is learning how nuch content covered and academic engaged minutes 
is sufficient and reasonable for making normal progress in reading and 
mathematics. We simply do not know how ouch content needs to be covered 
for an average student to make average pvogreiss. 

There is a great need for descriptive data on the percentage of engaged 
time which is obtained in different settings, as well as on the setting 
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variables which may contribute to engaged time. Such descriptive research 
is quite different from the observation systems of the 60 's because in this 
case the variable, academic enp.aged time. Is a reasonable proxy for student 
achievement . 

Normative studies . Aside from McDonald's study, there is little 

normative data on academic engaged time. Descriptive data on engaged time 

in low SES classrooms wou'iid be particularly interesting, because this 

might help explain the low achievement scores and the apparent low content 

covered. Illustrative normative studies could include: 

engaged time per child, per day, per subject area, per grade. 

engaged time divided into activities within a subject area 
(e.g., reading, writing, listening) 

percentage of engaged time in different contexts (e.g», 
working alone, in teacher supervised seatwork). One 
issue would be replicatinp; the data reported above 
on some contexts being more engaging than others. 

percentage of engaged time in engaged time and 
activities in different types of classrooms such 
as formal and infernal. The approaches used by 
Gump (19 ) would be particularly valuable here. 

Preliminary results suggest thai: students ar^ engaged in academic activities 

about 75 percent of the a:iotted t±x?e in reading in formal classrooms and 

about 65 percent of the allottee^ time in informal classrooms. But such 

statements are tentative and need to replicated in a number of studies. 

We also need more analytic-descriptive ::tudies such as those of 

Kounin (3 975, 1976) in whica he attempted to determine the possible causes 

for different engagement in different settings. Kounin's idea of signal 

continuity influencing engagement migh:: be applied and extended to elementary 

school contexts. 
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Experimental studies . Experimental studies are vitally needed in this 
area. We don't know the numbe. of engaged minutes ot the amount of content 
covered per day which are reasonable, sufficient, or humane. This ?.ast 
point is particularly important for low achieving childre Children who 
are reading two or three years below grade level are going to need to spend 
a preat dzal of time on reading if they are ever to catch up. Such 
children may need to spend two or three hours a day in readii^g if they ar^ 
ever to make two year's progress in one. But 90 to 120 academic engaged 
minutes per day may simply be too much for these children. 

We have been writing ^s if all types of engaged time in reading are the 
same, and this seems unlikely. Certain types and sequences of practice may 
be more important than others. We have also written as if 100 minutes with 
the McGraw Hill Programmed Readers is the same as 100 minutes with Ginn 360 
or Economy materials at the same level. This may or may not be so. It may 
be that if students work on materials which are at moderate difficulty for 
them then reasonable materials are equivalent. This issue also merits study 

Student choice and individualization . Four types of student choice 
in individualization have been noted by researchers: 
choice of task or activity, 

choice of time and sequence for doing prescribed activities, 
choice of behavior ^e.g., where to work, when to leave seat), 
choice of how to think about the subject matter 
Based on present research, it is the choice of task setting that seems 
most dysfunctional; whereas choice of behavior seems most functional (perhap 
because it signals a flexible, albeit task-oriented teacher). The area of 
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choice of time and sequence is rather interesting for future research, and 
is being studied in the current work at the University of Pittsburgh, 

Wang (1976) conducted a number of small scale experiments in the context 
of Individually Perscribed Instruction classrooms in which students had 
choice as to when to do their teacher-assigned work, Under these conditions, 
these students — who also had prior training in working alone — increased both 
their attention and the number of units completed when they had choice as to 
when to do their work. This type of student choice merits research in con- 
texts other than IPI. Similarly, in a correlational stMdy of IPI classrooms, 
Leinhardt (1975) found that extensive choice was disfunctional, whereas 
limited choice (e.g., what topic to study, when they needed special tutoring) 
was functional. 

Thus, there is a need for more research on the types of student choice 
which might be functional, as well as on the prior conditions which facilitate 
student choice. 

Similarly, two types of individualization have been distinguished 
within recent research on teaching: 

students working on different subject matter, 

students working on different activities within the same 
subject matter. 

In the studies by Soar and by Stallings, the negative correlations 
between 'Variety of activities" and elementary student achievement occurred 
primarily for situations in which students were working on different 
subject matter (e.g., reading, science, dance) at the same time. It is 
suggested that this situation may be particularly difficult for a teacher 
to monitor. In contrast, students who are working on different activities 
within the same subject matter may be easier to monitor, and such a setting 
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may be more responsive to offering a student more than one set of materials 
or activities for mastering a skill. Thus, the two settings should be 
distinguished in future research. 

Seatvork and work groupings . As noted, seatwork is a relatively 
unstudied ia. The first task is descriptive, and could focus on the 
percent of allotted time for seatwork, and the percent of engaged time in 
different contexts . The major seatwork contexts, as developed by Marliave 
and others at the Far West Laboratory are: working with others vs working 
alone, self paced versus teacher paced work, and teacher supervised versus 
unsupervised. ("Teacher paced groups" would be discussion or recitiation 
groups). As one learns to rank these contexts accordin.^ to percent of 
academic engaged time, hopefully, we can proceed, as Kounin has done, to 
develop concepts to explain probable causes of these differences. 

A related set of descriptive-correlational tasks involve identifying 
effective teachers (much as the above researchers have been doing) and 
learning their techniques for structuring and supervising seatwork. Such 
topics include how these teachers prepare students for their tasks, how 
they structure the distribution of materials and the shift from one topic 
to another, whether the successful teachers "make rounds" or whether they 
work at their desks with an eye out for problems, and how these teachers 
respond to student requests for help and clarification. 

But the major problem lies in maintaining student engagement when they 
are not being supervised. Many teachers find it very difficult to both 
work with a small group of children and manage seatwork at the same time. 
The solution which successful teachers appear to have worked out is to 
supervise seatwork in a large group setting, and to thus avoid working with 



30 



smaller groups and individual students* Given the problems of managing 
28 children, this may be the best solution* But although large group 
grill, drill, and supervise is an effective arrangement, probably the most 
effective, we hope that other, reasonable alternatives would be available. 

Another approach would be to conduct experimental studies aimed at 
increasing student engagement during seatwork* Good is currently (1977) 
conducting an experimental study on seatwork, and the staff at the Far 
West Laboratory are running small studies on increasing student academic 
engaged time* Such studies are urgently needed* 

Seatwork materials . Some materials which students use when working 
privately (e*g*, readers, books, workbooks) are probably more engaging than 
others* Yet, we know little about them*. Developing or identifying 
engaging materials could be a major research area* 

Verbal interaction versus seatwork * A preplexing problem which is 
emerging is the value of time spent on verbal interaction as compared to 
time spent on particularly for middleclass students. In 2nd and 3rd grade 
math, Brophy and Evertson found that public discussion was more productive 
than seatwork; whereas McDonald in 2nd grade math found that seatwork with 
the teacher directing the students to task was most productive* In Ath 
grade math. Good and Grouws found seatwork to be most productive with all 
types of verbal questioning coming out negative; whereas McDonald found 
that in 5th grade math it was best to teach the class as a whole using 
explanation, discussion, and questions. Similar to McDonald, with their 
fourth grade sample, Solomon and Kendall found negative (but non-significant] 
trends for all types of teachers interactions during seatwork — including 
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giving requested help and unrequested help — whereas all types of interac- 
tions with the whole class had a positive (but non-significant) tread. In 
sum, the issue as to the relative merit or seatwork versus verbal activities 
is confusing. 

Adding to this confusion is Kounin and Gump's (1975) v/ork with pre- 
school children in which they found that construction seatwork activities 
were more engaging than discussion. They reasoned that this was because the 
siyials to the child were more continuous during the seatwork activities. 
It is dangerous J of course, to extrapolate work vjith preschool children 
to -mathematics at the 4th grade, but hopefully, we will use some of the 
concepts or conceptual thinking developed by Kounin in approaching this 
puzzle. 

We now need to reconsider the relative value of seatwork versus dis- 
cussion tor achievement gain in areas such as reading comprehension. As 
noted below, there is a strong correspondence between the controlled 
practice pattern used in ''discussion" and a similar pattern in the workbooks. 
Similar questions are asked in each case. But we don't know when there is 
value in conducting this practice verbally, and when it is more efficient 
to let the workbooks do it. One type or the other may be more effective 
than the other for different types of students. One problem with the 
teacher-led controlled practice is that all students may not participate, 
yet, others may gain more through the modeling of the more able students- 
But according?; to Kounin, discussion may not produce as much academic 
engaged time as supervised seatwork because the signals are more continuous 
in seatwork. The area is presently unclear. 
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Another issue is the overall value of discussion. One could argue 
that discussion aids reading comprehension gain because it requires the 
students to consider and reconsider concepts, and that it promotes active 
cognitive processing — skills which should enhance reading comprehension. 
This is the type of argument McDonald used to interpret his findings for 
5th grade reading. There is an appeal to this argument which merits 
increased exploration and research. 
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FOOTNOTES 

This paper is both a summary and expansion of three previous papers: 
Rosenshine (1976), Berliner and Rosenshine (1977) and Rosenshine and 
Berliner (1977). Many of the ideas were developed jointly with Dave 
Berliner. Many of the well received summary sentences in this paper 
were originally written by Berliner, and were taken from the Berliner 
and Rosenshine paper. The immediate parents of the concept of Academic 
Learning Time are Berliner, Charlie Fisher, and Rick Marlieve of the 
Far West Laboratory. The grandparents include John Carroll (1963) 
who originated the idea of "opportunity to learn,** (Husen, ?965i), 
and Nate Gage who insisted thar I code content covered on my 
dissertation study. 

The term direct instruction also has many parents and grandparei:ts. 
Fred McDonald and I independently same up with the term in publications 
in 1976, although the term has been used in many informal conversations, 
particularly with Berliner, for years before that. We have no idea 
who first used this term. In the trademark DISTAR which was first 
used in 1969, the D and I stood for direct instruction. 

I wish to give thinks to my friends and colleagues — Dave, Ken, Jean, 
Jane, Norma, Charlie, Garry, Sam, Marcienne, Nate, Lee, Tom, Carolyn, 
Mike, and Leu ~ who gave a little help as I proceed through my mid- 
career crisis; to Jere Brophy and Sig Tobias who selected me for the 
AERA presentation; to Jean Osborn, Bonnie Armbruster and Tom Good 
who contributed ideas and editing. 
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